Free energy and inference in living systems

Organisms are non-equilibrium, stationary systems self-organized via spontaneous symmetry breaking and undergoing metabolic cycles with broken detailed balance in the environment. The thermodynamic free-energy (FE) principle describes an organism’s homeostasis as the regulation of biochemical work constrained by the physical FE cost. By contrast, recent research in neuroscience and theoretical biology explains a higher organism’s homeostasis and allostasis as Bayesian inference facilitated by the informational FE. As an integrated approach to living systems, this study presents an FE minimization theory overarching the essential features of both the thermodynamic and neuroscientific FE principles. Our results reveal that the perception and action of animals result from active inference entailed by FE minimization in the brain, and the brain operates as a Schrödinger’s machine conducting the neural mechanics of minimizing sensory uncertainty. A parsimonious model suggests that the Bayesian brain develops the optimal trajectories in neural manifolds and induces a dynamic bifurcation between neural attractors in the process of active inference.


Introduction
Although there is no standard definition of life [1][2][3][4][5][6][7], the literature often states that a living system tends to reduce its entropy, defying the second law of thermodynamics to sustain its non-equilibrium (NEQ) existence. However, conforming to the second law of thermodynamics, adjudication between the entropy reduction and augmentation of an open system must depend on the direction of irreversible heat flux at the system-reservoir interface. Organisms are open systems in the environment; hence, they obey the second law by contributing to the total-entropy increase in the universe. The above confusion, perhaps, is rooted in Erwin Schrödinger's annotation, which metaphorically explains living organisms as feeding on negative entropy [8]. In the same monograph, Schrödinger continues to explicate that a more appropriate discussion for metabolism is to be addressed in terms of free energy (FE). He made this clarification because, in contrast to the Clausius entropy to which he was referring, thermodynamic FE always decreases during irreversible processes in any open system [9]. Many studies have been based on Schrödinger's insight into how biological systems can be explained by physical laws and principles. We examine the definition of life in terms of FE minimization.
Organisms maintain biologically essential properties, such as body temperature, blood pressure and glucose levels, which are distinct from ambient states. Living systems continuously exchange heat and material fluxes with the environment by performing metabolic work, which is subject to the energy balance described by the first law of thermodynamics. The second law posits that the entropy of an isolated macroscopic system increases monotonically with any spontaneous changes. Organisms and the environment together constitute the biosphere, which is isolated and macroscopic; thus, metabolic processes in organisms increase the total entropy. The second law affects organisms by limiting metabolic efficiency. The thermodynamic free-energy principle (TFEP) encompasses thermodynamic laws and provides qualitative and quantitative explanation of how living systems biophysically sustain homeostasis by minimizing FEs. Recent studies have addressed the modern metabolism perspective as energy regulation of multisensory integration across both interoceptive and exteroceptive processes [10,11]. This explains metabolism not only at the level of individual organisms, but also at the ecosystem and planetary levels [12,13], and emphasizes the energetics and power efficiency in brain performance [14][15][16]. By contrast, the ability of organisms to undergo allostasis, which predictively regulates homeostasis [17,18], or, more generally, their autopoietic properties [19], are unable to be explained by the TFEP. Allostatic ability is the main driver of adaptive fitness, the emergence of which cannot be solely attributed to a (bio)physical self-organization from a myriad of emergent possibilities in the primitive circuits of neuronal activities.
Organisms are under environmental constraints, and adaptive fitness, or natural selection, is the consequence of survival optimization in specific environments during evolution. Therefore, the FE minimization scheme requires a top-down or high-level computational mechanism that facilitates hardwiring of the allostatic capability.
The brain-inspired FE theory in neuroscience and theoretical biology suggests a universal biological principle in an axiomatic manner, the free-energy principle (FEP). The FEP provides the informational FE-minimization formalism, that accounts for the perception, learning and behaviour of living systems in the environment [20,21]. This principle has been also applied to other cognitive systems, such as artificial intelligence and robots [22][23][24][25][26][27][28]; however, our study primarily focuses on living systems and implications of the FEP in a biological context, emphasizing the embodied nature of inference [29]. According to the informational FEP (IFEP), all life forms are evolutionarily self-organized to minimize surprisal, which is an information-theoretic measure of the improbability or unexpectedness of the environmental niche of organisms. Informational FE (IFE) is a theoretical construct, rather than a physical (thermodynamic) quantity specified by the temperature, chemical potential, volume, etc. Informational FE mathematically bounds the surprisal from above; accordingly, the IFEP suggests that natural selection reflects minimization of IFE in an organism as a proxy for surprisal at all biological time scales. The IFEP employs Helmholtz's early idea of perception as unconscious inference [30]: an organism's brain possesses an internal model of sensory generation and infers the external causes of sensory data by matching them with prior knowledge. The active-inference framework following from the IFEP encapsulates motor control and planning beyond Helmholtzian perception as an additional inferential scheme [31,32]. The brain possesses the probabilistic internal model whose parameters (sufficient statistics) are encoded by brain variables in the NEQ stationary state; however, thus far, no physical theory has been developed for determining NEQ probabilities in the macroscopic brain. In practice, the IFEP assumes open forms, or some fixed forms, for the NEQ densities and implements IFE minimization. The Gaussian fixed-form assumption can be used to convert the IFE to a sum of discrepancies between the predicted and actual signals [33], which is known as prediction error in predictive coding theory [34]. Commonly, the transformed IFE objective is minimized by employing the gradient-descent method widely used in machine learning [35]. The resulting variational-filtering equations compute the Bayesian inversion of sensory data by inferring the external sources [36], known as recognition dynamics (RD) [20]. Recently, the IFEP was generalized in a manner that minimizes sensory uncertainty, which is a longterm surprisal over a temporal horizon of an organism's changing environmental niche [37]. Despite being a promising universal biological principle, the IFEP has led to controversy regarding its success as the universal principle and its distance between biophysical reality and epistemological grounds [38][39][40][41][42][43][44][45][46][47].
In this study, the two FE approaches are jointly considered to develop a unified paradigm for living systems: the TFEP does not describe the brain's ability to infer and act in the environment, whereas the brain-inspired IFEP is mainly a purposive (hypothesis-driven) framework lacking intimate connections to neuronal substrates and physical laws. Our goal is to link the two FEPs and propose a biological FEP that integrates the reductionistic base and top-down teleology in the brain. In addition, we unveil the attractor dynamics that computes allostatic regulation, perception and motor inference, in the brain, based on our proposed FE-minimization framework. A similar approach was reported in [48], in which formalisms underwriting stochastic thermodynamics and the IFEP were presented without addressing the direct link between the thermodynamic and informational FE. In addition, a unified Bayesian and thermodynamic view attempted to explain the brain's learning and recognition as a neural engine and proposed the laws of neurodynamics [49]. We also note another recent work that made the neural manifold models from a symmetry-breaking mechanism in brain-network synergetics, commensurate with the maximum information principle [50].
In brain architecture, enormous degrees of freedom of neuronal activities pose the classical negligence in a highdimensional problem; thus, the underlying neural dynamics appears to be stochastic. However, we argue that perception, learning and motor-inference in the brain is low-dimensional at the functional level, obeying the law of large numbers; accordingly, RD becomes deterministic, involving a limited number of latent variables. For instance, a few joint angles suffice for the brain to infer arm movement in motor control. In contrast, the emergence of deterministic RD is more subtle in perception and learning, which demands a systematic coarse-graining of stochastic neuronal dynamics. Our investigation facilitates the systematic derivation of Bayesian-brain RD in terms of a few effective variables, which we term Bayesian mechanics (BM); BM conducts the homeostasis and allostasis (that is, adaptive fitness) of living systems, conforming with the proposed biological FEP.
The concept of coarse-graining, or effective description, is ubiquitous in computational neurosciences [51][52][53][54][55][56][57]. Here, we review the recent research relevant to our work, which motivated the development of BM. Many previous studies of recorded neurons showed that population dynamics is confined to a low-dimensional manifold in empirical neural space, where trajectories are neural representations of the population activity [58]. In mathematical terms, the neural modes were defined as eigen-fields that span the neural manifold. The latent variables, or collective coordinates, were defined as projection of the population activity onto the neural modes [59,60]. Other theoretical models support the idea that long-term dynamics in recurrent neural networks gives rise to the attractor manifold [61], which is a continuous set of fixed points occupying a limited region of neural space. Consequently, the attractor dynamics and switching between different attractors were manifested [62], indicating a contextual change in neuronal representations [63,64]. Moreover, the manifold hypothesis is widely applied in machine learning to approximate high-dimensional data using a small number of parameters [65]. Experimental studies showed that a dynamical collapse occurs in the brain from incoherent baseline activity to low-dimensional coherent activity across neural nodes [66][67][68]. Synchronized patterns emerged when the featured inputs and prediction derived from prior or stored knowledge matched; in contrast, when there was a mismatch, the high-dimensional multi-unit activity increased. This observation also provided empirical evidence that neural signals reduce prediction errors, thereby minimizing the IFE. Based on the results described above, we suggest that the latent dynamics can be effectively described by a small number of coarse-grained variables in the reduced dimension. In this study, we formulate the BM of inferential regulation of homeostasis in living systems in terms of a few latent variables. The latent variables are determined as the brain activities and their conjugate momenta that represent the external, environmental and motor, states and online prediction errors, respectively. The sensory error at the peripheral level acts as a time-dependent driving source in BM, providing the neural mechanism for sensory, as well as motor, inferences. Our continuous-state formulation in continuous time may be useful for studying situated-action problems in which biological systems must make decisions even during ongoing sensorimotor activity [69].
The remainder of this paper is organized as follows. In §2, we describe the establishment of the TFEP from NEQ fluctuation theorems (FTs) when applied to living systems. Section 3 explains how stochastic dynamics at the neuronal level can be modelled and how a statistical approach can be used to determine the NEQ densities of neural states in the physical brain. In §4, we present the proposed biological IFEP minimizing long-term surprisal and establish its continuousstate implementation that yields BM in the neural phase space. Next, in §5, we numerically integrate BM and manifest the attractor dynamics that performs perception and motor inference in the brain. Finally, we summarize important outcomes of our investigation and provide the conclusions in §6. In appendix A, we present the dual closed-loop circuitry of active inference resulting from our model.

Non-equilibrium fluctuation theorems applied to organisms
FTs concisely describe stochastic NEQ processes in terms of mathematical equalities [70,71]. Although FTs were initially established for small systems, where fluctuations are appreciable, they also apply to macroscopic deterministic dynamics [72]. Here, we present FTs in an appropriate context of biological problems and propose that the FTs suggest a living organism is an NEQ system that maintains the housekeeping temperature, T (average 36:5 C in humans) within its body and employs metabolism isothermally to act against its environment.
To this end, among the various expressions of FTs, we use the NEQ work relation [71]: where β = 1/(k B T), with k B being the Boltzmann constant and T being the temperature as described below. The mathematical equality given in equation (2.1) is known as the Jarzynski relation [73]. Here, W is the amount of experimental work performed on a small system immersed in a thermal reservoir and ΔF is the induced change in the Helmholtz FE of the system. Accordingly, W − ΔF is the excess energy associated with each irreversible work process in the system, which is unavailable for a useful conversion. The bracket, 〈 · · · 〉, indicates the average over many work strokes, that is, work distribution subject to a protocol. The average must be considered because the experimental work performance on small systems fluctuates. The Jarzynski relation can be converted to an expression for entropy as follows. By applying 〈e −βW 〉 ≥ e −β〈W〉 to equation (2.1), which is known as the Jensen inequality [74], we obtain the inequality ΔF ≤ 〈W〉. This inequality is an alternative expression that can be used to apply the second law to isothermal irreversible processes of the system initially prepared in equilibrium with a reservoir [72]. Using the inequality, one can consider the change in the average total entropy: 〈ΔS tot 〉 = 〈ΔS sys 〉 + 〈ΔS R 〉, where ΔS sys is the change in the system entropy and ΔS R is the change in the reservoir entropy. The average associated with ΔS R , which is reversible by definition, can be further manipulated to obtain 〈ΔS R 〉 = −〈Q sys 〉/ T = (〈W〉 − ΔU)/T, where Q R = −Q sys is used in the first step, and then the thermodynamic first law is applied for 〈Q sys 〉; U is the internal energy of the system. Therefore, T〈ΔS tot 〉 = 〈W〉 − (ΔU − T〈ΔS sys 〉) = 〈W〉 − ΔF, which leads to the stochastic second law for the combined system and reservoir: The possibility of tightening the preceding inequality has been investigated among researchers by revealing a non-zero, positive bound, leading to thermodynamic uncertainty relations [75,76]. The unavailable energy associated with individual work processes amounts to the total entropy change, namely, bðW À DFÞ ¼ k À1 B DS tot under isothermal conditions. By applying the final identity to equation (2.1), the Jarzynski equality is cast to the integral form of entropy fluctuation: In the biological context, W is the amount of environmental work involved in the metabolism of a living system, such as the biological reactions of oxygenic photosynthesis and aerobic respiration [77,78]. The biological work is not controllable and thus, stochastic. The FTs describe the imbalance between energy intake and expenditure in an organism while maintaining the housekeeping temperature. The Helmholtz FE increment in the living system over a metabolic work cycle is limited by the average environmental work done on the organism. The resulting inequality from the Jarzynski relation can be written in the organism-centric form as where we set hWi ¼ ÀhWi and DF ; ÀDF, which now states that the work performance, hWi, of a biological system against the environment (e.g. via metabolism) is bounded royalsocietypublishing.org/journal/rsfs Interface Focus 13: 20220041 from above by the thermodynamic FE cost, DF . The preceding inequality reflects the limited efficiency of metabolic work in living systems. Rare individual processes that violate equation (2.4) may occur in small systems; however, such statistical deflection is not expected in a finite biological system with macroscopic degrees of freedom. The equality in equation (2.4) holds for reversible work cycles in inanimate matter, attaining thermodynamic efficiency at its maximum, but not in the metabolic processes of living organisms, which are irreversible. Our consideration of metabolic work may be generalized to the multi-level autocatalytic cycles suggested as the chemical origins of life [79]. Note here that we considered the temperature appearing in the Jarzynski relation as the body temperature of a specific biological system, unlike the usual implication of FTs; in the standard derivation of the Jarzynski relation [80], the temperature, T, appearing in the NEQ equality is, by construction, the reservoir temperature. The FT is generally intended for an irreversible process during which the system temperature may not be defined. However, the initial and end states must be in equilibrium so that the FE is meaningful. The subtlety lies in the fact that the end-state temperature may or may not be the same as the reservoir temperature for experiments performed in isolation after the initial equilibrium preparation. Living organisms are in an NEQ stationary state, maintaining a housekeeping temperature, T, that is distinct from the ambient temperature, to which they equilibrate only when ceasing to exist. Thus, organisms are viewed as isothermal systems, which are open to heat and particle exchange with the environment.
The NEQ work relation expresses the second law of thermodynamics as the mathematical equality in equation (2.1). The second law, in its biological context, renders the thermodynamic constraint on living organisms given by the inequality in equation (2.4), which reveals the inevitable (thermodynamic) FE waste produced during metabolic cycles. However, this inequality accounts for neither self-adaptiveness nor brain functions, such as perception, learning and behaviour. To address these essential features of life, researchers currently employ a hybridizing scheme, which first proposes how the system-level biological functions operate and then attempts to make connections to biophysical substances. Particularly, the Bayesian mechanism built into the IFEP provides a crucial component in this promising hybrid explanation of life, which is described in detail in §4.

Statistical-physical description of the nonequilibrium brain
The brain comprises a myriad of complex neurons; accordingly, its internal dynamics at the mesoscopic level must obey some stochastic equations of motion on account of classical indeterminacy. The relevant coarse-grained neural variables are local-scale population activities, or intra-area brain rhythms. In the following, we consider that the brain matter itself constitutes the thermal environment at body temperature for the mesoscopic neural dynamics. Below, we assume that the neural activity, μ, at the coarsegrained population level obeys the stochastic dynamics [81]: where the inertial term in the Langevin equation was dropped by taking the over-damping limit. Here, f may represent both conservative and time-dependent metabolic forces, and w represents random fluctuation characterized as a delta-correlated Gaussian noise satisfying the following conditions: where I is the noise strength. In one dimension (1D), for simplicity, the environmental perturbation and noise strength are physically specified, respectively, as [82] f ¼ where A is a conservative force acting on a neural unit with mass, m, neglecting time-dependent driving, T is the body temperature, and γ is the phenomenological frictional coefficient whose inverse corresponds to momentum relaxation time. The solutions to equation (3.1) describe the individual trajectories of random dynamical processes.
In general, coloured noises can be considered beyond the delta-correlated white noise by generalizing equation (3.1) to incorporate the non-Markovian memory effect: To ensure equilibrium at temperature T, the coloured Langevin equation must satisfy the fluctuation-dissipation theorem that accounts for the non-singular noise correlation [83]: A standard example of such coloured noise is the Orstein-Uhlenbeck memory kernel given by γ( where τ is the noise autocorrelation time. As an alternative to the Langevin equation (equation (3.1)), one may collectively consider an ensemble of identical systems displaying various values of the state, μ, and ask how the statistical distribution changes over time. After normalization, the ensemble distribution is reduced to the probability density, say p(μ, t), so that p(μ, t) dμ specifies the probability that an individual Brownian particle is found in the range (μ, μ + dμ) at time t. In the Markovian approximation, the change in the probability density is determined by the probability density at the current time, which is generally described by the master equation given in the continuous-state formulation as where w(μ 0 , μ) is the transition rate of the state change from μ to another μ 0 . We further assume that the transition occurs between two infinitesimally close states, μ and μ 0 , where μ 0 − μ = x ≪ 1, so that the transition rate sharply peaks at around x = 0 to approximate the value as w(μ 0 , μ) ≈ w(μ; x). Then, p(μ 0 ) can be expanded about μ to the second-order in x and all higher-order terms are neglected. Consequently, the master equation can be converted into the Smoluchowski-Fokker-Planck (S-F-P) equation [82]: where D 1 and D 2 correspond to the first two expansion coefficients in the Kramers-Moyal formalism, which are determined in the present case to be The S-F-P equation can be expressed in three dimensions (3D) as where r is the gradient operation with respect to the threedimensional state, m. In equation (3.4), the drift term, pf, accounts for conservative potential forces. In addition, the diffusion term, ÀDrp, appears, assuming spatial isotropy for simplicity and notational convention, where D 2 has been replaced with D. The S-F-P equation describes local conservation of the probability, pðm, tÞ, in the state space spanned by the state vector, m, which carries the probability flux, j, identified as jðm, tÞ ¼ pðm, tÞfðmÞ À Drpðm, tÞ: In steady state (SS), ∂p st /∂t = 0, where p st ≡ p(μ, ∞); accordingly, the divergence of the SS flux, j st ; jðm, 1Þ, must vanish in the S-F-P equation: If Brownian particles undergo motion in an isolated or infinite medium, j st should disappear on the local boundary because the total flux through the surface must vanish to ensure probability conservation. 1 Because the flux must be continuous over the entire space, the SS condition in equation (3.5) imposes j st ; 0 everywhere, reflecting the detailed balance between the drift flux and dissipative flux. In this case, the system holds in equilibrium, where life ceases to exist. The equilibrium probability can be obtained from the condition j st = 0, giving canonical Boltzmann probability as the result: where V(μ) is potential energy. The kinetic-energy term does not appear in p eq because the Langevin dynamics we consider are in the over-damping limit. However, for a finite open system, such as a living organism, the system's SS flux does not necessarily vanish on the local boundary; instead, it must be compensated by the environmental afferent or efferent fluxes to achieve steady state. Thus, for a living system, the detailed balance is not satisfied in the steady state [84,85]; that is, j st = 0. Instead, the vanishing condition of the divergence of the probability flux entails a necessary balance. The mathematical expression in equation (3.5) admits a non-vanishing vector field BðmÞ via j st ðmÞ ; r Â BðmÞ, ð3:6Þ which shows that the SS flux is divergenceless or, equivalently, solenoidal [86][87][88]. The life flux, j st , defined in this manner is unchanged when B is transformed to where L is a scalar function of the state, m. 2 From equation (3.6), the following generalized balance condition must hold locally on the boundary: The above modified detailed-balance condition supports the frequent interpretation of the force field, f st , as the gradient flow of the SS probability, p st [3,91]: where we introduced the scalar field, Q(μ), via for simplicity, Q was assumed to be isotropic as it was for the diffusion constant, D. The gradient flow is driven by entropy because the most likely equilibrium state of the combined system and environment is achieved by maximizing the total entropy; hence, it is an entropic force, conforming to the second law. Note that equation (3.6) mimics the Ampere law in magnetism [92]; the effective field B may be construed as an induced field by the static current, j st . Accordingly, the vector field, BðmÞ, can be determined by means of Note that the modified detailed-balance condition given in equation (3.7) is only a formal description for determining the NEQ density, p st , given SS flux, j st , or, equivalently, the environmental magnetic field, B, in equation (3.9). Precise determination of p st is an independent research subject, which may be non-Gaussian with a coloured autocorrelation.
In general, it is difficult to obtain an analytic expression for the NEQ probability density for open systems, except in low-density and/or linear-response regimes [93,94]. Because of further morphological complexity, it is practically intractable to derive the NEQ densities specifying the physical brain states. Accordingly, in the following, the neural states under continual sensory perturbation are assumed to be statistically described by time-dependent Gaussian densities, predicted from Gaussian random noises imposed on the Langevin description.

Latent dynamics of sensorimotor inference in the brain
Here, we present the BM for conducting Bayesian inversion of sensory observation in the brain under the proposed generalized IFEP. This idea was previously developed by considering passive perception [37] and only implicitly including active inference [95]. Here, we advance this formalism by explicitly introducing motor inference and planning in the generative models to fully conform to the active-inference framework.
The environmental states, q, cause sensory stimuli, w, at the organism's receptors through mechanical, optical or chemical perturbations, which are transduced in the brain's functional hierarchy in the form of a nervous signal. The sensory perturbations may be altered by the organism's motor manipulation, and we designate u to denote the motor variables responsible for such control over the effectors. A crucial point here is that the brain has access only to the sensory data and not their causes; accordingly, from the brain's perspective, both the environmental states, q, and motor variables, u, are external, that is, hidden. In terms of these where qðq, uÞ and pðq, u; wÞ are the recognition density (Rdensity) and generative density (G-density), respectively. The R-density is the brain's online estimate of posterior beliefs about the external causes of the sensory perturbation (it probabilistically represents the environmental states). The G-density encapsulates the brain's likelihood in beliefs about sensory-data generation and prior beliefs about the hidden environmental as well as motor dynamics (it probabilistically specifies the internal model of sensory-data generation, environmental dynamics and motor feedback). Note that whereas the R-density is the current estimate, the G-density contains the stored knowledge in the brain, which can be updated by learning. In this study, we generalize the R-density as a bi-modal probability of q and u, and Gdensity as a tri-modal probability of q, u and w. Note that a semicolon is used between the sensory perturbation, w, and hidden variables q and u in the G-density rather than a comma to emphasize their differential role in perception.
The explicit inclusion of the motor variable, u, in the R-density and G-density is a key advancement over the standard definition of IFE [33]. Now, using the product rule, pðq, u; wÞ ¼ pðq, ujwÞpðwÞ for the G-density in equation (4.1), we decompose the IFE to a form applicable in the biological context: where D KL is the Kullback-Leibler divergence [74]. Because D KL is non-negative, the following inequality holds, which underpins the IFEP described in §1: À ln pðwÞ F ½qðq, uÞ, pðq, u; wÞ, ð4:2Þ where −lnp(w) is the information-theoretic measure of surprisal. Here, it is important to notice the resemblance between the preceding inequality and that given in equation (2.4) from the TFEP. Under the IFEP, the organism's cognitive goal is to infer the hidden environmental causes of sensory inputs with feedback from the motor-behaviour inference. This goal is achieved by minimizing F with respect to the R-density, qðq, uÞ, which corresponds to the online adaptation of the sensory and motor modules in the brain. For instance, in the classic reflex arc, the proprioceptive stimulus evokes the activity of sensory neurons in the dorsal root, and the motor variable is engaged by the effector's active states of the motor neurons in the ventral root. The double procedures are involved in the minimization scheme to cope with the bimodal cognitive nature of sensory and motor inferences: (i) the internal model is updated to better predict the sensory perturbation and (ii) the sensory perturbation is modified by the agent's motor engagement to further reduce the residual discrepancy with the internal model. The former is termed as passive perception and the latter as active perception. However, the two inferential mechanisms do not separately engage, but act as a whole in the sensorimotor closed loop in the embodied brain, and are therefore jointly termed as active inference under the IFEP [31,32].
To draw a connection between the IFE minimization and neural correlates, it is practically convenient to use the fixed form for the unknown R-density [33], whose sufficient statistics are assumed to be encoded neurophysiologically by brain variables, that is, neuronal activities. Here, we write the R-density as qðq, uÞ ¼ qðqÞqðuÞ by considering the external variables q and u as conditionally independent. Furthermore, it is assumed that the factorizing densities, qðqÞ and q(u), are Gaussian; the means of the environmental states, q, and motor states, u, are encoded by the neuronal variables μ and a, respectively. Then, by performing technical approximations similar to those used in [33], we convert the IFE functional, F , of the R-and G-densities to the IFE function, F, of the neural representations μ and a, given sensory data, s. The sensory data or inputs are a neural representation of the evoked perturbation, w, at the receptors, observed by the organism's brain. Here, the homunculus hypothesis, the brain as a neural observer, is implicit, which assumes teleological homology between the environmental processes and brain's internal dynamics.
The result for the IFE function, up to an additive constant, is given as Fðm, a; sÞ ¼ À ln pðm, a; sÞ; ð4:3Þ here, the dependence on the second-order sufficient statistics, namely (co)variances of the R-density, was optimally removed. Consequently, the brain need only update the means in the R-density in conducting the latent RD. The mathematical procedure involved in equation (4.3) extends the Laplace approximation delineated in the review [33]. To complete the Laplace-encoded IFE, one must specify the inferential structure in the encoded G-density, p(μ, a; s). We facilitate probabilistic implementation of the generative model using the product rule: pðm, a; sÞ ¼ pðsjm, aÞpðm, aÞ, ð4:4Þ where the likelihood density, p(s|μ, a), is the brain's concurrent prediction of the encoded sensory data, s, from the neuronal response, μ, and motor manipulation, a. Assuming conditional independence between μ and a, the joint prior p(μ, a) can be further factorized as pðm, aÞ ¼ pðmÞpðaÞ, where p(μ) and p(a) are the brain's prior beliefs regarding the environmental-state changes and motor dynamics, respectively. Thus, the Laplace-encoded IFE has been specified solely in terms of the neural variables μ and a, given sensory data s, which is suitable for biologically plausible implementation of active inference in the physical brain. Sensory states are stimulated by environmental causes encoding sensory data, which, in turn, neurophysically drive the neuronal population dynamics in the brain. The population dynamics is complex and high-dimensional; however, the RD of the perceptual and behavioural inferences may be well described in lower-dimensional neural manifolds. Below, we set up the plausible dynamics of coarsegrained neural variables from classical indeterminacy, which constitute our generative models. First, we assume that sensory data, s, are measured by a neural observer according to instant mapping: where g is the generative map of the sensory data, and z is the observation noise. Note that the generative map encapsulates both the perceptual states, μ, and motor states, a, which conjointly predict the sensory data, s. We consider the sensory generative model as a continuous process of sensory prediction, g 1 (μ), by μ and error prediction, g 2 (a), by a via the effector alteration: ½s À g 1 ðmÞ À g 2 ðaÞ ; s À gðm, aÞ, where we set g(μ, a) = g 1 (μ) + g 2 (a). Second, we assume that the neural activity, μ, obeys neuronal dynamics as described in §3: where f is the generative function of the neuronal change, and w is the involved random noise. Third, we assume that the motor state, a, bears the motor-neuronal dynamics: where π is the generative function of the motor-neuronal change, and η is the noise in the process. The generative function, π, plays the role of the policy in machine learning [35]: the policy function, pða; u p Þ, encapsulates the internal model of motor planning in continuous time (see appendix A). The dependence of the generative models on the parameters θ g , θ f and u p enables incorporation of a longer-term neural efficacy, such as synaptic plasticity; below, we omit the parameter dependence for notational simplicity. For the neuronal generative equations, the continuous Hodgkin-Huxley model [37] or a more biophysically realistic model can be employed; however, our simple model in §5 suffices to unveil the emergence of BM. Noises in the neural generative models (equations (4.5)-(4.7)) indicate stochastic mismatches between the cognitive objectives on the left-hand side (l.h.s.) and their prediction through the generative functions/map. Accordingly, we consider that z, w and η neurophysically encode the probabilistic generative models p(s|μ, a), p(μ), and p(a), respectively, (equation (4.4)) in the neuronal dynamics. Furthermore, we assume that the random noises are continuously distributed according to the normalized NEQ Gaussian. Therefore, the Laplace-encoded likelihood, p(s|μ, a), and prior densities, p(μ) and p(a), in equation (4.4) take the following forms: here, N ðx À h; 0, sÞ ; expfÀð1=2sÞðx À hÞ 2 g= ffiffiffiffiffiffiffiffiffi 2ps p denotes a Gaussian density of stochastic variable x − h with variance σ about the zero mean 3 , and _ x denotes the time derivative of x, that is, dx/dt. The generative likelihood and prior densities in equation (4.8) are thought to be stationary solutions to the S-F-P equation or a more general non-Markovian extension, the biophysical derivation of which is beyond the scope of this work. Instead, we assume the time-dependent Gaussian probabilities effectively at zero temperature as physically admissible densities encoding internal models in the brain. Removing the assumption by rigorously deriving physical probabilities is a key theoretical demand in future studies. where we dismissed the term 1 2 lnfs z s w s h g [37]. Our specific construct of the IFE encapsulates motor planning explicitly in continuous time via the policy, π(a), in the generative models. Based on the Laplace-encoded IFE, the mathematical statement for the biological FEP is given as ð dt{ À ln pðsÞ} ð dtFðm, a; sÞ, ð4:10Þ where the l.h.s. is equivalent to the Shannon uncertainty, Ð dsfÀ ln pðsÞgpðsÞ, under the ergodic assumption, which is assured by the NEQ stationarity of living systems. The inequality (equation (4.10)) shows that the upper bound of sensory uncertainty can be estimated by minimizing the time integral of F over a temporal horizon. Accordingly, if we regard the integrand F as a Lagrangian, the systematic framework of the Hamilton principle can be employed to implement the minimization scheme [96]. Next, we cast equation (4.9) to a weighted summation of the quadratic terms: where m i may be considered as a metaphor for the neural inertial masses. The neural masses correspond to the predictive precisions in the standard terminology [33]; heavier neural masses lead to more precise predictions. The IFE F as a Lagrangian, conforming to classical dynamics, can be considered as a function of the instant trajectories of μ(t) and a(t), subject to the time-dependent force, s = s(t).
To exercise the Hamilton principle, we define the classical Action, S, as the time integral of arbitrary trajectories μ(t) and a(t) in the configurational state space: S½mðtÞ, aðtÞ; tÞ ¼ where t 0 is the initial time, and τ ≡ t − t 0 is the temporal horizon of the relevant biological process. The initial time can be chosen either in the past, that is, t 0 → −∞, or at present, that is, t 0 = 0. In the former, t is the present time, whereas in the latter, t is the future time. Hence, active inference of the living systems mathematically corresponds to varying S, subject to the sensory stream, to find an optimal trajectory in the configurational state space spanned by μ and a.
royalsocietypublishing.org/journal/rsfs Interface Focus 13: 20220041 Furthermore, it is advantageous to consider the brain's RD in phase space rather than configurational space; the phase space is spanned by positions and momenta. This is because the momentum variables are meaningful prediction errors in the brain's message passing algorithms; they are defined via the informational Lagrangian, F, as where p m and p a are the momentum conjugates corresponding to μ and a, respectively. Equation (4.11) reveals that the momenta, p m and p a , are indeed the prediction errors, 1 m and 1 h , weighted by the neural masses, m w and m z , respectively. The purposive Hamiltonian, H, can be obtained by performing the Legendre transformation H ; p m _ m þ p a _ a À F. After straightforward manipulation, we obtain the Hamiltonian function: which is a generator of time evolution in neural phase space.
The function H is specified in the cognitive phase space spanned by the four-component vector, C, in the single cortical-column formulation, whose components are defined as where C T is the transpose of C. Having determined the Hamiltonian, the Bayesian mechanical equations of motion (termed as BM) can be abstractly written in the symplectic representation as where the block matrix J is defined as Specifically, we unpack equation (4.17) and explicitly display the outcome: which are a coupled set of differential equations that are nonlinear, in general. The preceding equations (4.18)-(4.21) comprise the BM of the brain variables, which execute the RD of the Bayesian perception and motor inference in the brain. The BM was attained by applying the Hamilton principle, for which we adopted the Laplace-encoded IFE as an informational Lagrangian and derived the Hamiltonian to generate the equations of motion. Our latent variables are the neural representations (μ, a) and their conjugate momenta (p m , p a ); they span the reduced-dimensional neural manifold. The momenta represent the prediction errors neurophysiologically encoded by the error units in the neuronal population. Below, we describe two significant features of the latent dynamics, governed by the derived BM, subjected to the time-varying sensory input, s(t).
(i) Equations (4.18)-(4.21) suggest that the brain mechanistically executes the cognitive operation, which reflects Schrödinger's suggestion of the organism's functional operation as a mechanical work [8]. Our derived BM addresses the continuous-state implementation of IFE minimization in continuous time, which contrasts frequent discrete-time approaches [97][98][99][100]. We considered that biological phenomena are naturally continuous and, thus, continuous representations better suit perception and behaviour. (ii) BM in symplectic form (equation (4.17)) represents the gradient-descent (GD) on the Hamiltonian function.
However, under non-stationary sensory inputs, the multi-dimensional energy landscape is not static, but incurs time dependence. Accordingly, the presented BM naturally facilitates fast dynamics beyond the quasi-static limit implied by the usual GD methods. In addition, it does not invoke the concept of higher-order motions in the conventional framework [101]; accordingly, our theory is not limited by the issue of average flows versus the rate of change of the average [38].

Numerical study of Bayesian mechanics
In this section, we numerically illustrate the latent dynamics of the brain's sensorimotor system resulting from the Hamilton principle-based FE minimization formulation. For simplicity, we consider a homogeneous, but time-dependent, sensory input, such as non-stationary light intensity or temperature, at the receptors, which emits a motor output innervating the effectors that alter the sensory observation.
There are approximately 150 000 cortical columns in the mammalian neocortex, and each cortical column exhibiting a six-laminae structure may be considered as an independent sensorimotor system [102,103]. Our simple model features the double closed-loop circuitry delineated in appendix A within a single column, which constitutes the basic computational unit of canonical circuits in an actual large-scale brain network [104]. The generative map, g, and functions, f and π, are unknown; they may be nonlinear or even undescribable within ordinary mathematics. Here, we exploit the linear models assuming the generic structures: gðm, a; u g Þ ¼ u ð0Þ g þ u ð1Þ g m þ u ð2Þ g a, ð5:1Þ where u ðiÞ a (α = f, g, π) are the parameters that are to be learned and encoded as long-term plasticity in the neural circuits. We have included the term u ð2Þ g a in equation (5.1), which facilitates the additive motor-inference mechanism of the sensory data; additionally, u ð1Þ g and u ð2Þ g magnify or demagnify sensory prediction and motor emission by the internal royalsocietypublishing.org/journal/rsfs Interface Focus 13: 20220041 state, μ, and motor state, a, respectively; u ð0Þ g denotes the default belief in the prediction. The constant terms u p modulate the relaxation times to the targets. In addition to these seven parameters u ðiÞ a , there appear three neural masses, m a , in the BM unpacked in equations (4.18)- (4.21). Hence, the proposed parsimonious BM still encloses 10 parameters, which define a multidimensional parameter space to explore for learning. The learning problem was not pursued in this study but should be explored in future investigations. Here, we focus on the active inference problem, assuming that the optimal parameters were already learned or amortized over the developmental and evolutionary timescales; these parameters are assumed to be shared for generating present and future sensory data.
By substituting the generative functions given in equations (5.1)-(5.3) into equations (4.18)- (4.20), the BM of the state vector, C, can be concisely expressed as where the relaxation matrix, R, is The first term on the r.h.s. of equation (5.7) describes the homogeneous solution for an initial condition of Cð0Þ, and the second term is the inhomogeneous solution driven by the source, I ðtÞ, manifesting the history-dependent feature. The solution represents the brain's cognitive trajectory in action while continuously perceiving the sensory inputs, s(t). In the long-time limit, t → ∞, we mathematically predict that the trajectory in the state manifold will fall onto either a fixed point, spiral node or repeller, satisfying _ C st ¼ 0 or a limit cycle about a centre satisfying _ C st ¼ ÀivC st , where ω is an angular frequency characterizing stationarity 4 . The details of the solution's approach to a steady state will be determined from the eigenvalue spectrum of the matrix R and time-varying feature of s(t). We denote the eigenvalues and eigenvectors by λ( ≡ iω) and ϕ, respectively, and set up the eigenvalue problem: The trace and determinant are invariant under a similarity transformation; accordingly, the ensuing eigenvalues must satisfy: X a l a ¼ trðRÞ ¼ 0, ð5:8Þ Y a l a ¼ detðRÞ f u ð1Þ f u ð2Þ g u ð2Þ g : ð5:9Þ The eigenvalues form the Lyapunov exponents in the finitedimensional manifold and characterize the dynamical behaviour of the state vector near an attractor. Because of the multi-dimensionality of the parameter space, it is not ideal to extract the eigenvalue properties analytically from the trace and determinant conditions. Accordingly, informative constraints on the parameters must be determined on the heuristic basis. In this study, we numerically searched for parameters that led to pure-imaginary eigenvalues, thereby entailing stationary attractors.

Numerical result I: spontaneous dynamics
We first consider the spontaneous dynamics of the brain evolved from the particular solution in equation (5.7) with null sensory inputs in our proposed BM. The formal representation for the spontaneous trajectory, C sp ðtÞ, can be obtained by direct integration as where the constant vector, C c , is specified as where I sp is the inhomogeneous term solely from the internal driving sources without the sensory inputs, that is, s = 0 (see equation (5.6)).
In figure 1, we depict the trajectories generated assuming a set of parameters in the neural generative models (equations (5.1)-(5.3)) as 5

ðu ð0Þ
g , u f , u ð0Þ p Þ ¼ ð0, 10, 10Þ, f Þ ¼ ð2 e ip=2 , À 1Þ and ðu ð2Þ g , u ð2Þ p Þ ¼ ðe ip=2 , e ip=2 Þ: In addition, the neural inertial masses were assumed to have values of ðm z , m w , m h Þ ¼ ð1, 1, 1Þ: The major numerical observations are as follows. The brain's spontaneous trajectory occupies a limited region in the state space around a centre, C c , which describes a dynamic attractor forming the brain's resting states before sensory influx occurs. The centre is specified by the internal parameters, that is, the generative parameters and neural masses. We numerically checked that the position of C c varies with the values of neural masses and the brain's prior belief on the hidden causes of the sensory input and motor state. We also confirmed that the size of attractors is affected by the generative parameters and neural masses.

Numerical result II: passive recognition dynamics
To demonstrate passive perception, we exposed the resting brain to a static sensory signal; that is, we inserted s = constant royalsocietypublishing.org/journal/rsfs Interface Focus 13: 20220041 in equation (5.6). In this case, the formal solution equation (5.7) can be reduced to where, on the r.h.s., the first term specifies the homogeneous transience of the initial resting state, Cð0Þ, second term, C c , denotes the centre of attractors, and last term describes the dynamic development from the inhomogeneous source, I ðsÞ. In contrast to the spontaneous attractors, the location of the centre depends on the sensory input, s: We performed numerical integration and obtained the stationary attractor in the presence of static sensory inputs. Thus, we confirmed that the attractor behaved similarly as in the spontaneous case, but with a shift of the centre because of the non-zero sensory stimulus. The outcome is presented in figure 2. Figure 2a shows a typical attractor in the two-dimensional state space, which is evolved from a spontaneous state shown in figure 1. In addition, in figure 2b, we show the change in the cognitive intensity, jC c ðsÞj 2 , with respect to sensory inputs, s, which is defined as Given a sensory stimulus, we numerically observe that the cognitive intensity is weaker for a larger inertial mass. The neural inertial masses represent the inferential precision in the internal models; accordingly, the result shows that less cognitive intensity is required when the internal model is more precise in perceptual inference. The cognitive intensity may be used as a quantitative measure of awareness or attention in phycology. Our intensity measure is closely related to neuroimaging analysis [105], where the neural response to sensory inputs was analysed as the energy-level change associated with information encoding.

Numerical result III: active recognition dynamics
To cope with active perception, we considered the nonstationary sensory input, s(t), that renders the time-dependent driving I (equation (5.6)) in the latent dynamics: the sensory receptors are continuously elicited, and the brain engages in online computation to integrate the BM. For numerical purposes, we assumed the salient feature of sensory signal, s(t), as a sigmoid temporal dependence: where t m indicates the time when the sensory intensity reaches the midpoint and k adjusts the stiffness of transience in approaching the limiting value, s(t) → s ∞ . The sigmoidal sensory inputs are depicted as a function of time in figure 3a.
We numerically integrated equations (4.18)-(4.21) assuming the same initial state selected for the data shown in figure  2a, subject to the sensory stream presented in figure 3a. In figure 3b, we illustrate the imaginary part of the motor state, p a (t), in continuous time, which is the online outcome of active inference of the sensory input. For illustrational purposes, we adopted the sigmoid shape for the temporal dependence with a saturated value of s ∞ = 100, stiffness of k = 0.2, and mid-time of t m = 250. The results suggest that the motor state aligns with the sensory variation and successfully infers the sharp change in the sensory input around t = 250.
In addition, figure 4 presents the attractor dynamics at several time steps exhibiting state transition, dynamic bifurcation, from a resting state, Cð0Þ, to a cognitive attractor, CðtÞ, over time [106]. The numerical computation reveals the initial development of the NEQ attractor with passage of time shown in figure 4a and figure 4b, which corresponds to the inferential outcome of the lower part of the sigmoid influx depicted in figure 3a. The intermediate attractor in figure 4b repeats the spontaneous attractor presented in figure 1 because the sensory input is nearly null apart from the negligible fluctuation in the present model. As time elapses from figure 4b to figure 4c, the cognitive state begins to escape from the first attractor and build the second attractor. Eventually, with passage of time shown in figure 4c and figure 4d, the dynamic transition between two attractors completes over a relaxation time period, say, τ. At time t > τ, the stationary attractor can be described by the expansion where iv a ; l a and f a are the eigenvalues and corresponding eigenvectors of the relaxation matrix, R, respectively. The expansion coefficients, c a , are specified by the initial condition, Cð0Þ. The centre of mass of the attractor, C c , is specified by R À1 I 1 , where I 1 is the source vector I with the saturated sensory input, s ∞ . The shift of the centre between two stationary attractors is shown in figure 4d.
The concrete example presented above fully accommodates the active inference of a living agent inferring the sensory signal's salient feature and performing feedback motor-inference in the double closed-loop cognitive architecture (see appendix A). Although the illustration accounts for a single sensorimotor system, our formulation can also handle multiple modalities of sensory inputs posing multisensory perception problems. Notably, the time-dependent royalsocietypublishing.org/journal/rsfs Interface Focus 13: 20220041 sensory influx, s(t), makes the BM non-conservative, which, from a dynamical-systems perspective, serves as a bifurcation parameter. Our numerical illustration of the dynamic transition from a resting state to a cognitive attractor is relevant to recent studies of cognitive control of behaviour in psychiatry [107,108] and stability of conscious states against external perturbations in patients with brain injury [109,110].

Summary and conclusion
This study is based on the consensus that living systems are self-organized into an NEQ stationary state that violates the detailed balance while sustaining physiological and bodily properties. In a biological context, the thermodynamic second law implies that there is inevitably uncompensated energy in an organism's metabolic processes of maintaining its homeostasis in the environment. More precisely, the amount of metabolic work is bounded from above by the thermodynamic FE expense. Efficiency is important in any irreversible phenomena exhibiting the arrow of time, and by extension, in brain work. We applied modern FTs to a biological agent as an open system and clarified why the concept of the FE is more appropriate than entropy when discussing the question of What is life? The thermodynamic and neuroscientific FEPs were evaluated based on their respective mathematical inequalities, implying the FE bounds as variational objective functions for minimization. Consequently, we revealed the disadvantages of both principles in accounting for cognitive biological systems and proposed an integrated thermodynamic and Bayesian approach to the biological FEP as a self-organizing principle of life.  royalsocietypublishing.org/journal/rsfs Interface Focus 13: 20220041 The brain states of higher organisms can only be realistically described probabilistically because of the enormous neuronal degrees of freedom and morphological complexity. And at the core of the biological FEP are the likelihood and prior densities, making up the G-density, which are thought to be the NEQ probabilities of the physical brain variables. The G-density was Laplace-encoded by the brain variables that were assumed to be the sufficient statistics of the R-density. This study argues that brain dynamics at the constitutional level are stochastic because of classical negligence, for which time-asymmetric Langevin equations were employed. The broken time-reversal symmetry was attributed to the fact that biological systems are open to the environment. To statistically describe the brain states, we further used the Markovian approximation in state transitions and adopted the S-F-P equation to determine the probability densities of the continuous brain variables. We viewed the S-F-P equation as a local balance equation for probability and argued that its steady-state solutions furnish the NEQ densities. The probability flux appearing in the S-F-P equation does not vanish at the brain-environment interface, which reflects that a detailed balance will not be reached in the SS limit, and thus, no standard fluctuationdissipation theorem is available in the NEQ brain. Instead, the SS flux resembles the Ampere law in magnetism, resulting from the modified detailed-balance condition and supporting the gradient flow of the NEQ probabilities.
We presented the brain as Schrödinger's mechanical machine operating predictive regulation of physiology and adaptive behaviour of the body. The BM at the system level is deterministic, indicating that the brain, as a macroscopic physical system, obeys the law of large numbers entailing dimensionality reduction. In addition, thermal fluctuations from body temperature do not have significant effects on the brain's low-dimensional functions; in other words, the brain is cognitively in its ground state at effective zero temperature. The IFE was specified in terms of the latent brain variables that probabilistically encode the environmental and motor states. As aforementioned, the encoded probability densities were assumed to be SS solutions to the S-F-P equation or more realistic ones. Central to our study was the idea that the encoded, online IFE in the brain is a Lagrangian, defining the informational action. Based on Hamilton's principle, we found that the brain deterministically conducts allostatic regulation by completing the double closed-loop dynamics of perception and motor behaviour. We employed a simple model for non-stationary sensory influx and illustrated the development of optimal trajectories in the neural phase space: we numerically observed that the brain undergoes a dynamic transition from a resting state to the  stationary attractor, which corresponds to the online inference of the environmental causes in continuous time. The proposed BM may apply to any generic cognitive processes at the interoceptive, exteroceptive and proprioceptive levels.
In conclusion, organisms' adaptive sustentation cannot be described within thermodynamic laws and the ensuing TFEP, for which the brain-inspired IFEP provides a promising avenue. The IFEP, however, uses teleological informationtheoretic models and then considers the neural bases of those models. To establish an integrated framework of the operational principle of life, two rationales of FE minimization and Bayesian inference were hybridized, and the BM directing the brain's latent dynamics of active inference was derived. Consequently, the brain's perception and motor inference in higher organisms were revealed to operate effectively as Schrödinger's mechanical machine. In addition, we numerically illustrated the attractor dynamics that develops online during a sensory stream in the low-dimensional neural space.
Data accessibility. This article has no additional data. Conflict of interest declaration. I declare I have no competing interests. Funding. No funding has been received for this article. Acknowledgements. The author is grateful to J. Kang for providing assistance with the mathematica programming.
Endnotes 1 In an isolated or infinite medium, the net flux through the entire surface must vanish to ensure probability conservation, i.e. j net ; Þ j st Á da ¼ 0, where da is the outward, infinitesimal area element. Accordingly, j st ¼ 0 at every point on the surface. 2 The freedom to choose the vector field, B, without affecting the physical quantity, j st , is known as gauge symmetry. Recently, researchers attempted to determine the implication and utility of the gauge transformation in neuronal dynamics in the brain and emergent functions [89,90]. 3 Here, we use σ, not σ 2 , to denote the variances only to be consistent with the notations in an earlier publication [33]. 4 In this study, we distinguish between the concept of a stationary state and a steady state: a steady state is the general term indicating the limiting state as t → ∞, whereas a stationary state is the specific steady state where an oscillatory time dependence remains. 5 These parameter values were selected in an ad hoc manner through numerical inspection to produce a dynamic attractor; therefore, the latent dynamics of the cognitive vector, C, is evolved in the extended, complex-valued phase space in the present manifestation.

Appendix A. Dual structure of perception and motor inference
Here, we describe a significant feature of our derived BM capturing the dual nature of the sensory and motor inference in the neocortex [111], and briefly discuss its relevance to other control theories. Figure 5 depicts the double-loop architecture of the neural circuitry emerging from the attained BM. The environmental cause, q, registers the sensory data, s, at the peripheral interface (receptors or input layers), and the brain conducts the variational Bayesian inference that conjointly integrates the double closed-loop dynamics of sensory perception (A) and motor control (B). Note that the neural units ðm, p m , a, p a Þ are connected by arrows for excitatory driving and by lines guided by filled dots for inhibitory driving. Loop (A): the state unit, μ, in neuronal population predicts the input, s, based on the internal model, g 1 ðmÞ ¼ u ð0Þ g þ u ð1Þ g m. The error signal, ξ z (μ) = s − g 1 (μ), weighted by the accuracy, m z , of the model, innervates the state-error unit, p m , in the population. The error unit assimilates the discrepancy and sends the feedback signal to the state unit. Then, the state unit updates its expectation and predicts the sensory reality again, which completes the passive perceptual loop. Loop (B): the motor (effector) unit, a, alters the sensory input, s, according to the generative protocol, g 2 ðaÞ ¼ u ð2Þ g a, to promote accurate sensation of the data. The error signal, m z ξ z (a) = m z (s − g 2 (a)), acts as a control command to call for an adjustment in the motor-error unit, p a . Then, the adjusted motor-dynamics transmits the feedback signal to the effector state to further modify the sensory reality, which completes the active perceptive loop. The double closed-loop dynamics concurrently continue until an optimal trajectory, CðtÞ, is fulfilled in the neural hyper-phase space, which corresponds to optimizing the informational classical action, S, defined in equation (4.13).
Our Hamiltonian formulation renders the sensorydriving term, s − g(μ, a), to appear explicitly in the BM (see equations (4.20) and (4.21)). Its role is similar to the unsupervised updating rule in the reinforcement-learning framework [35]; specifically, it resembles the continuous control signal in the optimal control theory described by the Hamilton-Jacobi-Bellman equation [112]. The sensory-discrepancy signal not only affects the prediction error, p m , in the state prediction (equation (4.20)), but also the prediction error, p a , of the motor inference (equation (4.21)); this interrelation provides the neural mechanism for adaptive motor feedback via equation (4.19). The momenta in our formulation are termed the costates in the deterministic optimal control theory.
Also, the policy, π, in equation (4.19) accounts for the online motor behaviour, which prescribes action planning and can accommodate a situated decision [69]. In the discrete-state formulations, the policy is defined as a sequence of actions or decisions in discrete time [99,113], where the authors incorporate the necessary state transitions directly in the definition of FE. On the contrary, our continuous-time theory defines the policy as continuous planning, which we model as the generative function of motor inference. The time-dependence of policy generates the history-dependent response of the brain's cognitive state; see equation (5.7), in which the time, t, can be either at present or in the future. When the dynamic perception is coupled to categorical decision making, the royalsocietypublishing.org/journal/rsfs Interface Focus 13: 20220041 mixed continuous-discrete approaches may shape the active inference problems [114].
Finally, in this work, we employed a set of simple and specific generative models (equations (5.1)-(5.3)) for a concrete numerical illustration. In practice, however, the employed models can be readily generalized. For instance, one may consider an action-dependent generative function, f (μ, a; θ f ), which will make the state dynamics (equation (4.6)) subjected to action. Similarly, the policy function pða; u p Þ in equation (4.7) may also be considered contingent on states μ. Further investigations using more realistic models are required to learn the implication and utility of our dual closed-loop dynamics, related to the standard control theories.